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Executive Summary 


This report describes the experiences of over 600 undergraduates who were taking STEM courses with 
in-person class meetings that had to shift to remote instruction in spring 2020 because of COVID-19. 
Survey data were collected through a national, random-sample survey conducted May 13 to June 1, 
2020. 


Compared to students taking courses in non-STEM subjects, students taking STEM courses reported 
lower levels of satisfaction with their courses both before and after the move to remote instruction. For 
both kinds of courses, course satisfaction levels were much lower after courses moved online. Aspects 
of the course experience that most students thought were worse online than before were opportunities 
to collaborate with other students, interest level of the course content, and feeling included as a 
member of the class. 


Internet connectivity issues were serious enough to interfere with students’ ability to attend or 
participate in their STEM course at least occasionally for 46% of students, with 15% of students 
experiencing such problems often or very often. Almost a quarter of students taking STEM courses (23%) 
reported experiencing hardware or software problems serious enough to impact their ability to attend 
or participate in their STEM course at least occasionally with 7% reporting that they had such problems 
often or very often. 


A large majority of survey respondents reported some difficulty with staying motivated to work on their 
STEM courses after they moved online, with 45% characterizing motivation as a major problem. A 
majority of STEM students also reported having problems knowing where to get help with the course 
after it went online, finding a quiet place to work on the course, and fitting the course in with other 
family or home responsibilities. Major challenges to continuing with the course were more prevalent 
among women, students identifying as a race/ethnicity group minoritized in STEM, students taking their 
course at a community college, and those from lower-income households. 


Overall, students who reported experiencing a greater number of major challenges with continuing their 
course after it went online expressed lower levels of satisfaction with their course after COVID-19. An 
exception to this general pattern, though, was found for students from minoritized race/ethnicity 
groups, females, and lower-income students. Despite experiencing more challenges than other students 
did with respect to continuing their STEM courses remotely, these students were more likely to rate the 
quality of their experiences when their STEM course was online as just as good as, or even better than, 
when the course was meeting in person. 


Only 14% of students taking STEM courses were very satisfied with how well they were learning in their 
course after it moved online. But most students did not attribute their dissatisfaction with aspects of 
their STEM courses after COVID with poor instructor preparation or limitations inherent in online 
learning. Moreover, online STEM courses that used a larger array of recommended online teaching 
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practices had higher student satisfaction levels than those that used fewer of them. The practices most 
closely associated with greater course satisfaction were sending personal messages to students about 
how they were doing in the course, using real-world examples to illustrate course content, and 
assignments calling for reflection on the state of one’s learning. 
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Side-by-side photographs of two young women illustrate Nicholas Casey’s April 4, 2020 New 
York Times article “College Made Them Feel Equal. The Virus Exposed How Unequal Their Lives 
Are.” On the left pale colors of beige and gray predominate as Isabel Caning stands in front of 
her family’s vacation home in Maine. On the right, we see the bright colors and painted palm 
trees on the food truck operated by the family of her classmate Tatiana Lathion. Both college 
seniors had played on the same lacrosse team, lived in the same college dorm, and were 
recruited to be undergraduate teaching assistants for the same course at Haverford College. 
When Haverford closed its campus for the coronavirus, Caning carpooled to the family home 
in Maine where she could continue her studies in a quiet place; Lathion went to Florida where 
she had to turn her attention to helping her parents find and prepare food to keep the family 
business afloat. Anita Isaacs, professor for the course for which both students assisted, 
summed up the change in perspective after COVID-19, “It’s as though you had a front-row 
view on American inequality and the ways in which it was disguised and papered over.” 


Background 


The hasty shift to delivering college courses through remote instruction in spring 2020 was difficult for 
both faculty and students. Faculty had to shift their courses online in a matter of days, and many were 
called upon to use technologies they had never worked with before. Students had to figure out where 
they would live, how they would support themselves, and how they could connect to their courses, all 
while missing the interaction and support of their peers and instructors. Experts in online teaching 
practices point out that the spring 2020 course experiences should not be construed as representative 
of online learning as it occurs in a course designed for this modality and delivered by instructors trained 
to teach this way (Hodges et al., 2020; O’Keefe, Rafferty et al., 2020). Nevertheless, as higher education 
institutions, faculty, and students look forward to fall 2020 and beyond, the “COVID experience” has 
much to teach us about barriers to student participation in online course activities and student 
perceptions of online instruction. We should not lose the opportunity to extract insights from this 
experience, which revealed the magnitude of underlying equity gaps as well as strengths and 
weaknesses in higher education’s preparation for delivering instruction online at scale. Gathering and 
synthesizing these lessons can inform future practice, not only in online courses but also in courses 
combining elements of digital and in-person instruction. 


Survey Sample and Method 


Digital Promise and Langer Research Associates (https://langerresearch.com) developed the Survey of 
Student Perceptions of Remote Teaching and Learning to capture the experiences of undergraduates 
taking courses that transitioned to online instruction in response to the COVID pandemic. The survey 
explores the nature of college courses as they were taught after the COVID-19 outbreak, the 
pervasiveness of various challenges undergraduates faced after the transition to remote instruction, and 
course features associated with higher levels of student satisfaction. Data analyses compared 
experiences of students from low-income, under-represented race/ethnicity and gender groups, or rural 
backgrounds to those of students without these characteristics. 
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A word on terms used in this report... 


Inequality versus inequity 

The choice of the term “inequality” for the title of this report was intentional and reflects the distinction between 
the terms “equality” and “equity” as we now understand them. Equality is about giving everyone the same thing or 
treating everyone in the same way. Equity refers to a standard of moral or ethical fairness that takes into account 
the fact that the exact same treatment or portion may not be enough to support full participation by every 
individual. Giving equal resources to everyone can result in inequity because those resources can be insufficient for 
some students to thrive (for example, English learners or those with a visual impairment may not be able to profit 
from the print textbook their classmates are using). Education systems have been undergoing self-examination with 
respect to inequities built into their business-as-usual practices. During the COVID-19 pandemic, however, many 
colleges struggled to provide all their students with even the basics of equal access to instruction. The loss of 
campus internet and computers meant that some students had what they needed to continue with their coursework 
while others did not — hence, inequality. 


Online versus remote learning 

Under normal circumstances, online courses are designed for delivery over the internet, and the faculty and 
students participating in them have chosen this modality. What happened during the spring of 2020 was a hasty and 
unanticipated shift of courses using face-to-face instruction to delivery online. Those with extensive experience 
designing and delivering online courses cautioned researchers and education critics against viewing the COVID 
experience as a test of online learning and suggested using other terms, such as “emergency remote teaching,” to 
describe what was going on (Hodges et al., 2020; O’Keefe et al., 2020). While acknowledging the validity of this 
point, we used the term “online learning” in the student survey because of its greater familiarity to students. We 
have carried through with use of the term “online” in this report to correspond to wording of the survey items. 
Occasionally, we do use the term “remote” here, but without intending any distinction from the instruction 
elsewhere labeled “online.” 


What kind of students? 

We recognize that the way individuals and groups of individuals are described is important, and that labels that 
government data collections and research publications have used in the past are increasingly viewed as problematic 
in lumping very disparate groups together (for example, describing people of Chinese, Filipino, and Hmong descent 
all as “Asian”), bear connotations about what is normal or standard (implicit in terms like “subgroup,” “non-White,” 
or “low-income”), and highlight one characteristic of an individual while ignoring others. At the same time, our 
research sought to capture the STEM learning experiences of different kinds of students in order to address equity 
issues, and our options for describing students were limited by the way in which background information was 
collected for the survey panel (e.g., using federal Community Population Survey items for race and ethnicity) and by 
the requirement to have large enough numbers of students with a common characteristic to produce a reliable 
estimate of the group’s response. For this latter reason, we can report reliable survey findings for only two 
race/ethnicity groups (Hispanic and non-Hispanic White). We did not have a large enough sample of Asian students 
to support reporting for that group by itself (yet alone for its constituent groups of Asian ethnicities), nor was there 
an adequately large sample of Black students taking STEM courses. To use survey responses from the entire sample 
and be able to examine differences in course experiences associated with race/ethnicity group, we created two large 
categories-one consisting of students who described themselves as Black, Hispanic, Native American or Alaskan 
Native, Pacific Islander, or a combination of backgrounds including one or more of these, and one of students who 
described themselves as non-Hispanic White or Asian or a combination of backgrounds involving these two. These 
two broad categories are admittedly coarse, but they do capture the divide between those race/ethnicity groups 
that are found in STEM majors and careers at rates lower than their prevalence in the U.S. population and those that 
participate in STEM at rates equal to or greater than their population prevalence. We refer to these groups as 
“minoritized in STEM” and “non-minoritized in STEM” while acknowledging that Asian students, included in our non- 
minoritized group for purposes of discussing STEM education, are minoritized in other contexts. 
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This survey was administered to a random national sample of 1,008 undergraduates, age 18 and older, 
who were taking college courses for credit that included in-person class sessions when the COVID 
pandemic hit and had to finish the course by learning at a distance. Respondents had the option to take 
the survey in Spanish or English. The survey asked respondents to choose one of their STEM courses that 
had shifted online to describe in answering survey items. Students who were taking more than one 
STEM course were asked to respond for the STEM course they thought was most important for their 
future. Students who were not taking any STEM courses were asked to respond for the non-STEM 
course they considered most important for their future. 


Sixty-two percent of the national survey sample of 1,008 had been taking one or more STEM courses 
with in-class meetings that had to go online during the COVID-19 pandemic, and this report describes 
their responses. Forty-two percent of the sample described themselves as majoring in a STEM field. 


Survey results have a sampling error of 3.6 percentage points for the full sample and 4.6 points among 
students who took a STEM course. 


Data were collected May 13 to June 1, 2020, with sampling and data collection by Ipsos Public Affairs via 
its online, probability-based KnowledgePanel®, which provides Internet survey access to randomly 
recruited participants. Survey findings for the entire sample of 1,008 students are reported elsewhere 
https://www.everylearnereverywhere.org/resources/suddenly-online-national-undergraduate- 


survey/ ). 


A fuller description of the survey methodology and sample weighting appears in Appendix A. Appendix B 
contains responses to each survey item from students taking STEM courses. 


Findings 
Who were the students taking STEM courses in 2020 before the COVID-19 pandemic? 


Participants in the lpsos Knowledge Panel complete an extensive personal profile when they join the 
panel. This procedure enabled us to examine characteristics of undergraduates enrolled in STEM courses 
and to contrast them with those of students who were not taking any STEM courses without having to 
ask what some might have construed as intrusive questions on our survey. 


Gender. When asked to complete a survey, females are consistently more likely than males to 
participate (Curtin et al., 2000; Moore & Tarnai, 2002), and that was the case with our undergraduate 
survey as well. Before applying sampling weights, 64% of survey respondents were female (compared to 
54% of all undergraduates according to the American Community Survey). Population weights based on 
the American Community Survey were applied to obtain nationally representative data from the survey 
sample. After sampling weights were applied, larger proportions of male (67%) than of female (60%) 
survey respondents were taking one or more STEM courses with in-person meetings before all classes 
went remote in spring 2020. 


Next we examined the gender break down of the 620 students within the sample of 1,008 who were 
taking one or more STEM courses in spring 2020. The greater proportion of females in the 
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undergraduate population and the greater propensity of males to take STEM courses counterbalanced 
each other: Our weighted survey sample of STEM coursetakers was 50% male and 50% female. 


Race/Ethnicity. Of non-Hispanic White students responding to the survey, 66% said they were taking a 
STEM course as were 61% of Hispanic students and 54% of Black students. The weighted sample of 
survey respondents taking STEM courses was 54% non-Hispanic White, 22% Hispanic, and 12% Black, 
with the remainder comprised of Asian and other race/ethnicity groups. Only the Hispanic and non- 
Hispanic White subgroups were large enough to generate precise statistical estimates. At 55, the 
unweighted sample size for Black students is too small to provide reliable estimates for this group by 
itself. For purposes of statistical testing of differences related to race and ethnicity, two broader student 
groups were formed: Students from minoritized race/ethnicity groups under-represented in STEM 
(consisting of Black, Hispanic, Alaska Native or Native American students and respondents who 
described themselves as mixtures of these backgrounds) constituted 36% of the STEM survey sample, 
with students from groups better-represented in STEM (consisting of non-Hispanic Whites and Asians) 
comprising 64%. 


Age. Students taking STEM courses spanned a wide age range. Forty percent were under age 21; 26% 
were 21 or 22 years old; 13% were age 23-25; and 20% were age 26 or older. 


Income Level. Less than a third (28%) of the students in the STEM survey sample came from households 
with annual incomes of less than $50,000; 30% from households with incomes between $50,000 and 
$100,000; and 42% from households with annual incomes of $100,000 or more. For students who were 
not taking STEM courses, the comparable figures were 32%, 28%, and 40%. 


Urbanicity. Among undergraduates who were taking a STEM course, 37% said they lived in a location 
that was classified as urban; 52% in one classified as suburban; and 11% as rural. The relatively small size 
of the latter subgroup limits what can be inferred about rural students taking STEM courses based on 
this survey. 


Region. Of respondents taking a STEM course, 37% said they lived in a location classified as the South; 
26% as the West; 20% as the Midwest; and 17% as the Northeast. 


Our survey included questions asking students to indicate the subject of the course they were describing 
on the survey, the kind of higher education institution offering that course, how many online courses 
they had taken in the past, and whether or not they were majoring in a STEM field. 


Type of Institution. More than two-thirds (70%) of the undergraduates taking STEM courses were taking 
them in four-year colleges. Students from minoritized race/ethnicity groups were more likely than non- 
Hispanic White students to be taking their STEM courses at two-year colleges: 43% of students from 
minoritized race/ethnicity groups were taking their STEM course at a two-year college compared to 21% 
of students from non-minoritized groups. More specifically, 45% of Hispanic students and 39% of Black 
students in the survey were taking their STEM course at a community college. Community college 
attendance was also more prevalent among students age 26 or older, students from urban or suburban 
households compared to those from rural homes, and students from Western states compared to those 
from other geographic regions. Students from higher-income households (with annual incomes of 
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$100,000 or more) were less likely than those from less affluent homes to be taking their STEM courses 
at two-year colleges (21% compared to 35%). 


Among students attending four-year colleges, 79% said their institutions were public and 21% said they 
were private. (Very few students attending two-year colleges said their institutions were private.) 


Course Subject. Of the STEM courses described by students, 17% were biology courses; 15% were 
mathematics courses; 13% computer science or technology; 12% engineering or engineering technician; 
10% chemistry; 10% health professions; and the remainder were courses in other STEM subjects, none 
of which was described by 10% or more of the sample. To obtain adequately large subgroups for 
subsequent analyses, course subjects were clustered into four broad fields: biology; mathematics; 
computer, information or data science; and other STEM subjects. Students who described a 
mathematics course were more likely to be taking their course at a two-year college (41%) than were 
students describing a STEM course in other STEM subjects (with proportions studying at a two-year 
college ranging from 23% for computer science and related subjects to 28% both for biology and for 
other STEM subjects). 


Major Field. Almost two-thirds (63%) of students taking STEM courses in spring 2020 said they were 
majoring in a STEM field; 29% said they were majoring in a non-STEM field; and the remainder were 
undecided or said they were taking courses for a program that did not necessitate a major. 


Class Size. The STEM courses students described for the survey had fairly small classes. Fifty percent of 
students reported that their course had fewer than 35 students, 29% said it had 35-75, and 21% said it 
had more than 75 students. It is likely that some students were describing the enrollment in their 
section of a course that had a larger total course enrollment. 


Prior Online Course Experience. Most of the undergraduates taking STEM courses in spring 2020 had 
limited prior experience taking a course online. Forty-three percent had never taken a course online 
before, and 20% had taken only a single online course. A larger proportion of community college 
students (45%) than of students taking their course at four-year colleges (33%) had taken two or more 
online courses in the past. 


There were some modest differences between the demographic characteristics of students taking STEM 
courses compared to undergraduates who were not taking STEM courses. The only statistically 
significant difference between the two groups, however, was that students who were taking STEM 
courses tended to be more male (50% versus 43%) than those who were not taking a STEM course. 


The remainder of this report describes survey findings for students taking STEM courses and is organized 
around the study’s six primary research questions: 


e How did students’ experiences with their STEM courses “post-COVID” compare with experiences 
prior to the move to remote learning? 

e What challenges did students experience in learning STEM online? 

e Which challenges were more prevalent among low-income, minoritized race/ethnicity group, 
and rural students compared to other students? 

e What was the nature of online learning in STEM courses during the COVID-19 pandemic? 
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e What online instruction practices do students perceive as contributing to their engagement and 
learning? 

e Towhat extent did different types of prior digital learning experiences or other variables 
mitigate the challenges associated with the transition to remote STEM learning? 


How did students’ experiences with their STEM courses “post-COVID” compare with experiences prior 
to the move to remote learning? 


Students were asked about their satisfaction with their STEM course both before and after it moved 
completely online, with response options of very satisfied, somewhat satisfied, somewhat dissatisfied, 
and very dissatisfied. As shown in Figure 1, almost half of the students describing STEM courses (47%) 
said they were very satisfied with their course before it went fully online. Student satisfaction with their 
STEM course after it went online was much lower, with only 15% of students describing themselves as 
very satisfied with their post-COVID STEM course experience. Another 42% described themselves as 
somewhat satisfied with the STEM course they were describing after it went online. 


Student satisfaction with their course after it went fully online was lower for students describing STEM 
courses than for students describing non-STEM courses. Twenty-seven percent of students describing 
non-STEM courses were very satisfied with their course after it went fully online compared to 15% of 
students describing STEM courses. This difference in course satisfaction levels for STEM and non-STEM 
courses predated the shift to online instruction: Sixty percent of students describing non-STEM courses 
said they were very satisfied with their course before it went online compared to 47% of students 
describing a STEM course. 


Nevertheless, the majority of students (57%) describing STEM courses said they were at least somewhat 
satisfied with their course after the shift to remote instruction. In summary, while fewer than 1 out of 

6 students was clearly happy with their STEM course during COVID, a majority were at least more 
positive than negative (the aggregate of the very satisfied and somewhat satisfied respondents). 


Figure 1. Satisfaction with their course before and after the move to remote instruction 


Before 
8 Very Satisfied 
& Somewhat Satisfied 
& Somewhat Dissatisfied 
After am Very Dissatisfied 


0 25 50 75 100 
Percent of Students 
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The survey also asked students to rate their satisfaction with multiple dimensions of their course after it 
went online, as shown in Figure 2. The area in which satisfaction suffered most after COVID was how 
well students thought they were learning: Only 14% of respondents said they were very satisfied with 
how well they were learning post-COVID. In contrast, 23% of students describing non-STEM courses 
reported being very satisfied with their learning in their course after it went online, a significant 
difference. 


Figure 2. Student satisfaction with quality of four dimensions of their online course 


Instructor's Preparation 


Quality of Course Content BB Very Satisfied 


a Somewhat Satisfied 
ii Somewhat Dissatisfied 
Fa] Very Dissatisfied 


Quality of Instruction 


Your Learning Overall 


0 25 50 75 100 
Percent of Students 


Students were also asked to compare six aspects of their course experience before and after the course 
went online. These six facets of student course experience were included on the survey because prior 
research suggests that they relate to student outcomes including course grades, college retention, and 
persistence in STEM fields (National Academies of Sciences, Engineering, and Medicine, 2019). There is 
also research evidence that some of these experiences are especially consequential for groups under- 
represented in STEM (women in most STEM fields, students from low-income backgrounds, and 
students with disabilities as well as those from minoritized race/ethnicity groups). 


A student’s sense of belonging in a course or field of study is associated with college retention and 
persistence in the field (Strayhorn, 2012). Ample research shows that women and students of color have 
a lower sense of belonging in STEM courses of study (Good et al., 2012; Johnson, 2012; Smith et al., 
2013), with female students of color least likely to feel a sense of belonging in STEM fields other than 
biology (Rainey et al., 2018). In a related vein, studies of persistence in college STEM programs suggest 
that some students perceive STEM faculty as unavailable or intimidating, leading to a low level of 
interaction with faculty, an experience associated with leaving STEM as a field of study (Daempfle, 
2004). Peer interactions and strong interpersonal relationships can increase students’ sense of 
belonging (Ramsey et al., 2013; Smith et al., 2013), and students’ sense of belonging in STEM is 
associated with student retention and persistence in the field (National Academies of Sciences, 
Engineering, and Medicine, 2019; Smith, Lewis, Hawthorne, & Hodges, 2013; Walton & Cohen, 2011). 
Perceptions of instructor responsiveness also appear to be important. Fouad et al. (2010) found that 
math and science teachers who were quick to help students who needed assistance had students with 
greater interest in pursuing careers in those fields. Another line of research suggests that instructors can 
contribute to students’ sense of belonging and content mastery by increasing the amount of structure in 
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a course and the transparency of their expectations for course assignments (Winkelmes, 2013). This 
practice appears to be especially helpful for students from minoritized race/ethnicity groups (Eddy & 
Hogan, 2013; Winkelmes et al., 2016). Similarly, opportunities to collaborate with fellow students in 
peer instruction and in project-based learning have been found to enhance learning for all students but 
appear to have a particularly positive impact for students from minoritized groups (Freeeman et al., 
2014). First-year courses that engage students in teams working together on real-world engineering 
problems, for example, have been found to increase student persistence in engineering (Fortenberry et 
al., 2007; Lichtenstein et al., 2014). Finally, the research literature has highlighted a mismatch of 
personal and cultural values held by students from minoritized race/ethnicity groups and those in STEM 
fields as a factor in limiting diversity in STEM (Eccles, 2009; Stephens et al., 2012). Students who begin 
STEM majors and then switch out of them often cite a lack of interest in the content (Emekalem, 2019). 


For purposes of keeping the student survey brief and easy to interpret, survey items corresponding to 
students’ course experiences related to these constructs were written in simple, everyday language. 
Transparency around course expectations was operationalized as “understanding what was expected of 
you in the course,” for example. Two aspects of instructor responsiveness were probed: “the instructor‘s 
knowledge of your strengths and weaknesses” and “the availability of help with the course content.” 
Sense of belonging in the course was characterized as “your feeling included as a member of the class.” 
The extent to which the course made STEM content interesting was expressed as “keeping you 
interested in the course content.” Opportunities for collaborative learning and forming connections with 
peers were stated as “opportunities to collaborate with other students on course work.” Figure 3 shows 
students’ comparison of these experiences in their STEM courses as taught before and after going 
completely online. 


Figure 3. Comparison of student experiences in their course before and after the shift to remote 
instruction 


Understanding what was expected 
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Specific aspects of the course experience that half or more of the students perceived to suffer after 
moving to remote instruction were: 


e Opportunities to collaborate with other students on course work (67% thought opportunities 
were worse or much worse online), 

e Keeping you interested in the course content (59% thought worse or much worse online), and 

e Your feeling included as a member of the class (54% thought worse or much worse online). 


On the other hand, a majority of students thought the course was about the same, better, or much 
better after going online in terms of: 


e Their understanding of what was expected in the course (64%), 
e The instructor’s knowledge of their strengths and weaknesses (56%), and 
e = The availability of help with the course (55%). 


Differences among race/ethnicity groups in how remote instruction affected students’ course 
experiences are discussed later in this report. 


Clearly most students were less happy with their STEM course after it moved online. Importantly, 
though, several aspects of the data suggest that most students did not lay the blame for the 
deterioration of their course experience on their instructors or on inherent limitations of the online 
modality. As shown in Figure 2, their instructor’s preparation for teaching their course online was the 
dimension of their course students were least likely to rate negatively. Asked about change over time 
during the period of online instruction, 29% of students said their instructor’s online teaching skills got 
better over time during the COVID pandemic compared to 16% who thought it deteriorated over time. 
In response to another survey item that asked respondents to choose between the unplanned move to 
online instruction and inherent limitations of online learning as the source of problems with their course 
as it was implemented post-COVID, more students attributed problems to the former (48% compared to 
39%) with 13% saying they didn’t experience any problems with the course after it moved online. 


What challenges did students experience in learning STEM online? 


The survey included an open-ended item asking students to describe the biggest challenge they 
experienced in continuing their STEM course after it moved entirely online. 


Many student responses to this open-ended question focused on staying motivated when they did not 
have the structure of regular class routines. 


Staying on top of the work was hard for me. If | was in my bed it was hard to get up and do the work 
before the deadline. — Black female student at a public four-year institution 


Mustering the motivation to watch recorded lectures. Often my professors would assign lectures that 
were longer than the assigned duration of the class, and lectures were not made available at the 
proper times. Online classes felt much more like work than they ever did on campus.— White male 
student at a public four-year institution 


Discipline to get things done was the hardest to overcome during this time. — Black male student at a 
public four-year institution 
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Many students described distractions or stresses in their home environments as contributors to this lack 
of motivation to work on their course. 


Being a mother to four children | had no away time from them until late at night. It is just very 
difficult to work from a computer for each class when having small children. — Hispanic female 
student at a private four-year institution 


| think it was harder to feel like the content was worth spending time on. Having the entire internet 
in front of me can be incredibly distracting and it makes doing the work seem less appealing. It was 
also much easier to be able to use other mediums to learn the material, making it seem a bit more 
unnecessary.—Black female student at a public four-year institution 


Vivir en un hogar latino, donde primero son los quehaceres antes que la escuela. [Living in a Latino 
home, where chores come first before school.] — Hispanic female student at a public four-year 
institution 


A related issue cited by many students was time management, another area in which loss of their 
regular routine added to the challenge. 


Time management. Because | didn’t have live online classes and then | could listen to lectures 
whenever, so! was unmotivated to listen to the lectures on my own time. — Other race female 
student at a public four-year institution 


Trying to stay focused and on top of things without having the same daily schedule and regular in- 
person chats. It was a lot easier to get left behind and for once in my academic career, | actually got 
behind a lot on work. — White male student at a public four-year institution 


Another theme found in many of the open-ended responses was loss of social engagement. Students 
reported missing the presence of their instructor and peers while they learned. Student responses such 
as those below suggested that there was an affective component to this challenge, with students 
expressing nostalgia for lost connection and closeness. 


Because of the separation from the professor, | had a hard time keeping motivated because the class 
wasn't designed for online learning. — White male student at a private four-year institution 


Losing the ability to ask classmates for help and having group study sessions. — Hispanic female 
student at a public four-year institution 


Not being able to get into contact with the professor as easily as being in person and knocking on 
their office door. — White male student at a private four-year institution 


Other students writing about lost engagement pointed to more cognitive aspects, highlighting loss of 
the opportunity to get immediate feedback from their instructor concerning their understanding or 
performance and of missing the chance to deepen their understanding by discussing course content 
with peers. 


Math is difficult to piece together solo, especially higher-level math. While the instructor lays out the 
steps very well, it isn't possible to ask questions immediately since the lectures are pre-recorded 
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video. There's a delay, and it doesn't help as effectively as immediate response. — White female 
student at a public four-year institution 


Not being able to ask questions at the time they form and having to wait before getting clarification 
on confusing segments. — Hispanic male student at a public four-year institution 


If | missed a step in a math problem | couldn’t get answers right away. I’d have to send an email and 
wait. — Hispanic female student at a two-year college 


During in person lectures | was able to ask my peers and my teacher for help with something | was 
struggling with and they would be able to show me how to do it. When class went online | was no 
longer able to ask for help the same way | could before, and it was harder to teach myself the class. — 
White female student at a public four-year institution 


Many STEM students also highlighted the loss of authentic hands-on experiences that were important to 
developing understanding or for the particular subject they were studying. 


There was not a good way to get help while working on any of the lab portions. Hard to know if you 
were doing things correctly with proper outcomes. — White male student at a public four-year 
institution 


Organic chemistry lab was much harder because you were writing a lab report on an experiment you 
obviously didn't do physically — White female student at a public four-year institution 


My clinical in-hospital experiences were moved to online. It was very hard to get anything out of it. | 
could not spend face to face time with patients or actually doing the work. — White female student at 
a two-year college 


Some of the students’ descriptions of their challenges suggested frustration with their instructors’ 
teaching techniques, lack of effort, or unfamiliarity with the online teaching tools they were using. 


Asking questions was much harder because the professor struggled with the application and 
frequently missed student questions and concerns. — Multi-racial female student at a public four-year 
institution 


[My] instructor would sometimes put on a video [for us] to learn and then walk away without 
realizing that he had turned off the sound so we couldn't hear it. He'd come back an hour later and 
then ask how it went. We tried to ping him to tell him the audio was off but he didn't give us a way 
to reach him so we watched it silently so it didn't get counted as we were absent but we got nothing 
out of the video. — Hispanic male student at a public four-year institution 


The teacher's inflexibility in group projects once the class went online and the lack of instruction on 
how to use new technologies such as Zoom, Google Slides, etc. It was expected for all students to use 
these technologies but no information was given for how to use them to their full capacity. It was 
more self-taught not only with the course but also with the technology needed to access and 
participate within the course.. — Black female student at a public four-year institution 
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In response to this question about their greatest challenge, a small proportion of students answered by 
saying they had adapted very well when the course moved online. 


None. | did very well in my courses. — Black female student at a public four-year institution 


| didn't feel like | had any challenges specific to the "online" aspect of the course. The greatest 
challenge was probably just that the course content was difficult. - Other race female student at a 
private four-year institution 


All in all it’s a great learning experience. We have all had to come out of our comfort zone and go 
with the flow. If we work real hard we can step out of the box and rise to the occasion. It’s not 
always easy but we have to try and our professors work with us and for most of us were all doing 
well. What can | say this is real life and we all are having to adapt. — White female student at a 
public four-year institution 


The survey also included structured-response items asking students to rate their experiences with a 
specific set of problems they may have faced in trying to continue with their courses after COVID. 
Consistent with responses to the open-ended question, the single problem reported by the largest 
proportion of students was staying motivated to do well in the course after it went online: 82% of STEM 
students cited this as a problem (45% said it was a “major problem” for them, and another 37% said it 
was a “minor problem”). As shown in Figure 4, other problems were less prevalent than motivation 
issues but still considered at least a minor problem for a substantial portion of students taking STEM 
courses. Not knowing where to go for help with the course was cited by 59% of students; finding a quiet 
place to work on the course online and fitting the course in with family/home responsibilities were both 
cited by 56%; feeling too unwell to participate by 46%; and fitting the course in with a work schedule 

by 31%. 


Figure 4. Severity of various problems with online course participation 
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Internet connectivity. Internet access they already had at home was the way the great majority of 
students (96%) accessed their STEM courses after COVID. Internet access at their school or near their 
home at another place arranged by their school was used by just 2% of students. Another 1% relied on 
Internet access in another public place for their course. 


Not all internet access is sufficient for digital learning activities, however. Videoconferencing, which was 
used in many courses, requires high bandwidth, for example. And even high-bandwidth connections can 
become slow and unreliable when multiple people in the same household are using them at the same 
time. Internet connectivity issues were serious enough to interfere with students’ ability to attend or 
participate in their STEM course at least occasionally for 46% of students, with 15% of students 
experiencing such problems often or very often. 


Poor internet connection made it difficult to submit tests and quizzes in a timely manner, and the live 
lecture would frequently buffer. — White female student at a public four-year institution 


Living on the reservation, no wifi, little help and understanding from instructors. — Other race female 
student at a public four-year institution 


Making sure my WiFi didn’t have disconnections especially during bad weather so | could submit my 
assignments in a timely matter. — Hispanic male student at a two-year college 


Device adequacy. Four out of five of students (80%) used a laptop computer to access their course after 
COVID. A desktop computer was used by 16%, a tablet by 3%, and a smartphone (which some have 
argued is inadequate for full participation in academic learning; see, for example, Anderson & Kumar, 
2019; Hampton et al., 2020) by 2%. The great majority of students in STEM courses (92%) were the sole 
users of the device through which they accessed the course, but 7% had to share their device with 
others (a lower proportion than the 14% found for respondents in non-STEM courses). A number of 
colleges and universities were active in distributing computing devices to students who did not have 
them this spring (Achieving the Dream, 2020; Flaherty, 2020), and 4% of survey respondents taking 
STEM courses said they were accessing their course through a device provided by their college. Another 
4% of respondents said the device they used was owned by someone other than themselves or their 
college. 


Almost a quarter of students taking STEM courses (23%) reported experiencing hardware or software 
problems serious enough to impact their ability to attend or participate in their STEM course at least 
occasionally with 7% reporting that they had such problems often or very often. 


Hardware and software problems tended to be experienced by the same students who had difficulties 
with their internet connection. About one in six students taking a STEM course had both hardware/ 
software problems and internet connectivity problems often or very often. 


Students who experienced a greater number of challenges with continuing their course after it went 
online also expressed lower levels of satisfaction with their course after COVID-19. We created an eight- 
point challenge scale by assigning a 1 for every one of the six challenges in Figure 4 that a student cited 
as a major problem plus a point each for having connectivity or software/hardware problems often or 
very often. Table 1 shows course satisfaction levels by the number of challenges a student experienced. 
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Table 1. Student satisfaction with course overall and with how well they were learning by number 
of challenges experienced. 


Number of Challenges Experienced as Major 
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Students reporting higher numbers of challenges with learning online were less likely to describe 
themselves as very satisfied and more likely to describe themselves as very dissatisfied with their STEM 
course post-COVID. Among students who did not report any major problems or technology issues, 25% 
described themselves as very satisfied and 4% as very dissatisfied with how well they were learning in 
their course after it went online. In contrast, among students reporting four or more challenges, only 7% 
described themselves as very satisfied and 52% described themselves as very dissatisfied with how well 
they were learning. 


Having difficulty maintaining motivation to do well in the course, not knowing where to find help with 
the course, and feeling too unwell physically or emotionally to participate were the specific challenges 
most closely related to overall course dissatisfaction. The proportion of students very satisfied with their 
post-COVID course was only 4% among those who cited motivation as a major problem compared to 
36% among those who said motivation was not a problem for them. The corresponding satisfaction 
rates were 4% for those who said they had a major problem finding where to get course help online 
compared to 24% among those who didn’t see this as a problem and 7% for those feeling too unwell to 
participate compared to 21% among those not having a wellness problem. 


Similar relationships were found between the number of challenges a student experienced and how well 
the student felt he or she was learning in the course, also shown in Table 1. 


Variability in the Experience of Challenges. The proportion of students reporting four or more major 
problems or challenges in trying to continue their course online was very similar for students taking their 
courses at two- and at four-year institutions (10% for the former and 12% for the latter). The proportion 
of students experiencing four or more challenges did vary, however, by a number of demographic 
variables as shown in Table 2 below. The proportion of women reporting four or more major problems 
was more than twice that of the proportion of men (17% versus 7%). Part of the explanation for the 
gender difference may relate to childcare responsibilities: Women were more likely to be responsible for 
taking care of a child during the COVID pandemic, and those taking care of a child were more likely to 
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experience four or more challenges (18% compared to 13% for those who either were not living with a 
child or who were in the same household but not responsible for childcare). Hispanic students were 
more likely to report having experienced four or more major problems (19% and 16%, respectively) than 
were non-Hispanic white students (7%). Students in the lowest of the three annual household income 
groups (annual income of less than $50,000) were more likely to report having had four or more major 
challenges (19% versus 10% for students in the middle-income group and 8% of those in the highest- 
income group). Students who had taken four or more online courses previously were less likely to report 
having four or more problems with their STEM course post-COVID (only 7% did so). 


Table 2. Number of major challenges experienced in continuing their STEM course online, 
by student group 


Number of Major Challenges 


Student Characteristic O-1 2-3 4 or more 
Challenge Challenges Challenges 


Which challenges were more prevalent among low-income, minoritized race/ethnicity group, and 
rural students compared to other students? 


Using respondents’ self-identifications in terms of race and ethnicity, we categorized students into 
groups large enough to permit statistical estimation. Here we focus on comparing the experiences of 
Hispanic students to those of non-Hispanic White students to better understand the disparate impacts 
of the transition to remote instruction. (There were too few students in other race/ethnicity categories 
to support reliable statistical estimates.) 


Staying motivated to do well in the course, which was the most prevalent challenge for non-Hispanic 
White students (with 45% considering it major) was similarly challenging for Hispanic students (44%). 
Not being able to find a quiet place to work, the second most widespread challenge overall, was 

somewhat more common among Hispanic, but did not differ significantly from the frequency among 
non-Hispanic White students. Twenty-three percent of Hispanic students said finding a quiet place to 
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work on the course was a major problem for them, compared to 18% of non-Hispanic Whites. Larger 
group differences were found for other challenges. Fitting the course in with home/family 
responsibilities, for example, was a major problem for 27% of Hispanic students compared to just 14% of 
non-Hispanic White students (p < .01). Feeling too unwell, physically or emotionally, to participate in the 
course was another challenge more prevalent among Hispanic students: 17% of Hispanic students 
reported feeling too unwell to participate as a major problem compared to 11% of non-Hispanic Whites 
(p <.10). More Hispanic students (27%) than non-Hispanic White students (15%) reported having a 
major problem knowing where to get help with the course after it went online (p < .01). These data are 
summarized in Table 3. 


Table 3. Frequency of major problems trying to learn online for selected race/ethnicity groups 


Hispanic Non-Hispanic 
Challenge (n = 135) White 
(n = 373) 
Staying motivated to do well in the course 


Not having a quiet place to work on the course 
Not knowing where to get help with the course 27%"* 


Conflicts with home/family responsibilities 27%** 
Feeling too unwell to participate 
Conflicts with work schedule 


Differs from non-Hispanic White group at p < .10 


* Differs from non-Hispanic White group at p < .05 
** Differs from non-Hispanic White group at p < .01 


Findings for the two large race/ethnicity categories (minoritized and non-minoritized) are shown in 
Table 4. The pattern of differences is the same as that shown in Table 3, but with the larger sample sizes 
more of them reach statistical significance. 


Unmasking Inequality 16 


Table 4. Frequency of major problems trying to learn online by race/ethnicity catrgory 
Challenge Minoritized? Non-minoritized” 
6 (n = 193) (n = 402) 
Staying motivated to do well in the course 
Not having a quiet place to work on the 24% 20% 
course 


Not knowing where to get help with the 25%** 14% 
course 
Conflicts with home/family responsibilities 25%** 


* Includes students who identify as Hispanic, Black, Native American, Alaska Native, Pacific Islander, or 
multiple groups including Hispanic. 


> Includes non-Hispanic White and Asian students. 
* Differs from non-minoritized student group at p< .05 
** Differs from non-Hispanic White group at p < .01 


‘ Differs from non-minoritized student group at p< .10 


In terms of technology access, students from minoritized race/ethnicity groups were more likely to 
experience internet connectivity problems often or very often (20% ) compared to students from non- 
minoritized groups (12%). Students from lower-income households (annual incomes under $50,000) 
were also more likely to experience internet connectivity problems often or very often (22%) compared 
to students from households earning $100,000 or more (11%). Surprisingly, students with homes in rural 
communities were not more likely than those in urban or suburban communities to report experiencing 
internet connectivity problems often or very often (13% versus 13% for urban and 16% for suburban 
locations), but the sample of rural students taking STEM courses was relatively small. 


Hardware or software issues were less prevalent than internet connectivity problems for the survey 
respondents overall, but showed a similar pattern of differences among race/ethnicity groups. 
Frequency of hardware and software problems was lower for non-Hispanic White students (5% 
experiencing them often or very often) compared to Hispanic and Black students (11% combined). 


Hardware and software issues varied by household income. Just 4% of STEM students from high-income 
households reported having these problems often or very often compared to 10% of students from 
lower-income households and 8% of students from middle-income households (annual income of 
$50,000—$100,000). 


1 For a discussion of the policy implications of these data, see the National Digital Inclusion Alliance report Limiting Broadband 
Investment to “Rural Only” Discriminates Against Black Americans and other Communities of Color. 
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Some of the responses of students to the open-ended question about greatest challenges also suggest 
that lower fluency in English and feelings of shyness or stereotype threat may have differentially 
affected some students when trying to participate online. 


My greatest challenge is trying to formulate a question in email. | sometimes struggle to explain 
what | am trying to ask via an email. It is easier to speak and explain my mind when | am verbally 
speaking to someone. — Hispanic female student at a two-year college 


Fue mas dificil participar en discussiones con la clase porque no era igual como cuando estabamos 
en persona. En persona, podemos levantar la mano para pedir decir algo, y podemos quebrarnos en 
grupos physicamente. En linea, eramos mas timidos en compartir nuestras opinones porque no 
queriamos interrumpir a otra persona. Tomabamos turnos, y hacia la discussion mas lenta. [It was 
more difficult to participate in discussions with the class because it was not the same as when we 
were in person. In person, we can raise our hands to ask to say something, and we can physically 
break into groups. Online, we were more shy in sharing our opinions because we did not want to 
interrupt another person. We took turns, making the discussion slower.] — Hispanic female student at 
a two-year college 


Attributes that put students at a disadvantage when trying to continue their course remotely (in terms 
of the number of challenges they faced) are often intertwined. For example, females, low-income 
students, and students from minoritized race/ethnicity groups were more likely to have responsibility 
for caring for a child during the COVID pandemic. Fifteen percent of female STEM students were 
responsible for childcare compared to 7% of males, and 20% of Hispanic students were responsible for 
childcare compared to 8% of non-Hispanic White students. Among those STEM students from lower- 
income households, 18% were responsible for taking care of children during this period compared to 
10% of those from middle-income households and just 6% of those from higher-income households. 
Similarly, students from minoritized race/ethnicity groups taking STEM courses were more likely than 
their White classmates to be working 31 or more hours a week during COVID (16% for Hispanic students 
compared to 8% of White students). 


What was the nature of online learning in STEM courses during the COVID-19 pandemic? 


Table 5 shows common features of STEM courses offered remotely because of the coronavirus After 
courses moved to remote instruction post-COVID, 70% held “live” (synchronous) sessions in which 
students could ask questions and participate in discussions. Recorded lectures were also common (71% 
of courses) as was the use of frequent quizzes and assessments (69%). 


Table 5. Features of Online Courses After COVID 


Percentage of Students 
Course Feature . 
Reporting 


Recorded lectures 


Pre-recorded videos, not of instructor 
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Grading. A major change in courses after COVID concerned course grading policies. Many colleges and 


universities extended the time period for choosing to take a course pass/fail or credit/no credit rather 
than for a letter grade to very late in the academic term. Some institutions even mandated that all 
courses use credit/no credit grading out of concern that differences in technology access or ability to 
participate in course activities would make grading unfair. Of the survey respondents who were taking 
their STEM course for a grade before COVID (92% of the sample), 61% reported being given the option 
of taking it either for a grade or pass/fail after COVID and 33% reported that they had no option but to 
continue taking their course for a letter grade after COVID. Seven percent of students reported that 
their STEM course became mandatory pass/fail after COVID. 


While inspired by a concern for being fair to students with limited technology access and other 
challenges to learning at a distance, the abolition or relaxation of grading policies may have contributed 
to students’ struggle to stay motivated to work on their course. When asked about their greatest 
challenge, several students made comments linking the grading system to their level of motivation. 


What online instruction practices do students perceive as contributing to their engagement and 
learning? 


The survey asked students whether their course after COVID included each of eight instructional 
practices chosen because they could be expressed in concrete terms on a survey and are prominent in 
the higher education practitioner literature (American College and University Educators, n.d.; Darby, 
2019; Luskin, 2019; Salcido & Cole, 2018) as contributors to more effective teaching and learning online: 


e Assignments that ask students to express what they have learned and what they still need to 
learn 

e Breaking up class activities into shorter pieces than in an in-person course 

e Frequent quizzes or other assessments 

e Live sessions in which students can ask questions and participate in discussions 

e Meeting in breakout groups during a live class 

e Personal messages to individual students about how they are doing in the course or to make 
sure they can access course materials 

e Using real-world examples to illustrate course content 

e Work on group projects separately from the course meetings 


Students’ comments when asked about their greatest challenge learning online underscored the 
importance of a number of these practices. 


Staying engaged/focused during lecture. Especially in a 1.5 hour class, motivation started to decline 
rapidly around halfway through. — Multi-race female student at a private four-year institution 


The videos my professors recorded of themselves giving the lecture was nice because it was actually 
coming from them instead of a random person on YouTube. — White female student at a public four- 
year institution 
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Not as engaging as in class instruction. The courses were too focused on teaching the book and not 
the real life examples. Being able to ask questions during the lessons was also absent as it was pre- 
recorded. — White male student at a public four-year institution 


When my genetics course was moved online | no longer had access to campus tutoring, and my 
professor did not hold office hours for students to ask questions and clarify course content. | was left 
to try and understand all of the material on my own, which was difficult. - White female student at a 
two-year college 


Nearly complete loss of motivation. Especially in classes that only had recorded lectures. It was 
worlds more manageable in classes where professors checked in periodically and where we had 
synchronous learning. But nothing can compare to in-person learning. — White female student at a 
public four-year institution 


Table 6 shows the frequency of each of the recommended instructional practices reported by STEM 
students overall and by students in various kinds of STEM courses. There were statistically significant 
differences by subject area in the proportion of students reporting the use of assignments having you 
reflect on what you had learned (p < .05), live sessions in which students could ask questions (p < .01), 
meeting in breakout groups during a live class (p < .01), and using real-world examples to illustrate 
course content (p < .01). 


Table 6. Percentage of students reporting each instructional practice, by course subject area 


Computer, 
Information 
All STEM or Data Other 
Subjects | Biology Science STEM? 
Assignments to express what 
they have learned and what 47% 37% 44% 57% 49% 
they still need to learn 
into shorter pieces 
assessments 


anal 
Live sessions in which 70% 59% 66% 71% 76% 
students could ask questions 

cigs 
rilifdualenucee. 
Using real-world examples to 65% 68% 56% 62% 70% 
illustrate course content 


a Because none of the other STEM subject areas had more than 30 students, they were grouped together as “Other 
STEM.” 
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When the relationship between experiencing each of these practices and a student’s satisfaction with 
the course was examined, we found that instructional practices with the largest effects on students' 
overall course satisfaction were: 


e Personal messages from the instructor on how the student was doing or to make sure they 
could access course materials. Sixty-four percent of those who received messages of this type 
were satisfied with the course overall, compared to 50% of those who did not. 

e The instructor's use of real-world examples to illustrate course content. Among students whose 
instructor did this in the online course, 65% were satisfied overall, compared to a 42% 
satisfaction level among those whose instructor did not use real-world examples. 

e Assignments requiring students to express what they had learned and what they still needed to 
learn. Satisfaction with online courses that included such assignments was 67% compared to 
49% for courses without this feature. 


The number of these practices students said their course included was tabulated for each respondent. 
The number of these online instructional practices incorporated in the course strongly predicted 
students’ satisfaction with their course after COVID. 


Figure 5. Satisfaction with Post-COVID course by number of recommended online practices used in 
instruction 


Satisfaction with 
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To what extent did different types of prior digital learning experiences or other variables mitigate the 
challenges associated with the transition to remote STEM learning? 


Online courses have become a much more prominent part of education over the past two decades 
(Tyton Partners & Babson Research, 2017), but the majority of students taking STEM courses that 
included in-person classes this spring had limited prior experience with online courses as noted above 
(43% had not taken any online course before and 20% had taken only one previous online course). A 
high level of satisfaction with their course after it moved online was more common among the 37% of 
STEM students who had taken two or more online courses in the past: 20% of them were very satisfied 
compared to 11% of those who had taken none, but this differences was not significant statistically. 
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A number of other variables also predicted students’ level of satisfaction with their course post-COVID. 
As one would expect, students who were very satisfied with their course before COVID were more likely 
to be at least somewhat satisfied with it afterward: 66% of those who were very satisfied before were 
very or somewhat satisfied after, compared to 61% of those who were just somewhat satisfied before 
COVID and 33% for those who were somewhat dissatisfied or dissatisfied with their course before it 
went online. 


Students’ ratings of their satisfaction with their STEM courses before the COVID-19 pandemic were 
higher for students taking their courses from two-year colleges than for those at four-year institutions 
(56% very satisfied before COVID at community colleges compared to 43% at four-year institutions). The 
college type satisfaction gap was much smaller and nonsignificant statistically for student ratings of 
satisfaction with their STEM courses after COVID-19, however (17% very satisfied in two-year colleges 
compared to 14% in four-year institutions). To understand this difference in the “pre/post” rating gap (a 
pre/post difference of 39 percentage points in two-year colleges compared to one of 29 percentage 
points for four-year institutions) we explored several possible explanations. Community college 
students, who were older and more diverse than the students taking STEM courses at four-year 
institutions, might have experienced a greater number of challenges during the COVID-19 pandemic. 
Alternatively, community colleges might have been less prepared than four-year colleges to move their 
courses online. 


Neither of these explanations for a bigger change in course satisfaction at community colleges is well- 
supported in the data, however. Students attending two-year colleges did not report more major 
problems than students attending four-year institutions did. Students at the two kinds of institutions did 
not differ either in terms of likelihood of experiencing frequent internet connectivity or 
hardware/software problems. One significant difference we did find for students attending the two 
types of higher education institutions is that those taking STEM courses at a two-year college were much 
more likely to simultaneously be responsible for the care of a child under 18 (20% for STEM students at 
two-year colleges compared to 4% for STEM students at four-year colleges). Students at two-year 
colleges were also more likely than those at four-year institutions to be working more than 20 hours a 
week during the pandemic (33% of respondents from two-year colleges compared to 14% of those from 
four-year institutions). Weighting each challenge equally in our tally of major problems experienced may 
have under-estimated the seriousness of stresses associated with providing childcare and working 
nearly full-time or more while also completing a STEM course online. 


In terms of institutional readiness to move their instruction online, several data points suggest parity 
between two- and four-year institutions from the standpoint of their student “customers.” Students at 
two-year colleges were not less satisfied with their instructor’s preparation for online instruction nor 
were they more likely to say the quality of their learning suffered after the move to remote instruction. 
Another approach to explaining the larger course satisfaction drop for two-year colleges is to look at the 
instructional practices employed in courses at the two kinds of institutions. We examined the proportion 
of courses exhibiting the recommended instructional practices by institution type. As shown in Table 7, 
most of the statistically significant differences by institution type favor the community college courses— 
with the prominent exception of assignment of group projects as part of the course after it went online. 


Unmasking Inequality 22 


Table 7. Frequency of recommended online instructional practices by institution type. 


Two-year Four-year 
colleges colleges 
what they still need to learn 
39%* 
80%* 


67% 
28% 
65%" 

Using real-world examples to illustrate course content 66% 
* Statistically different from the other institution type at p< .05. 


29 
63 
24% 
2 
2 
65 


A companion survey of faculty conducted by Tyton Partners during the same time frame 
(https: 
smaller proportion of two-year colleges had a center for teaching and learning or instructional design 


resources/time-for-class-covid-19-edition/) reported that a 


staff who could help faculty transition their course to the online environment. These supports may have 
helped faculty at four-year institutions preserve aspects of their courses that students like that were not 
captured by our survey items. 


It should be remembered, though, that students at two-year colleges were not less satisfied than those 
at four-year colleges with the post-COVID versions of their STEM courses. The difference resided in their 
higher course satisfaction level pre-COVID, which may well have reflected the value two-year college 
students place on instructor and peer contact in small classes. 


Level of satisfaction with their STEM course after it went online was related to several demographic 
variables. Female students were more likely than males to be very satisfied (18% versus 12%) with the 
online version of their STEM course (p < .05). There was greater satisfaction with their post-COVID STEM 
course among minoritized students, but this difference was driven by the responses of Black students 
(14% of both Hispanic and non-Hispanic White students were very satisfied). Students age 25 or older 
were more likely than younger students to be very satisfied with their course post-COVID (23% versus 
13%; p < .05). These differences will be explored further in the discussion section of this report. 


There was also some suggestion that satisfaction with their course post-COVID varied by subject matter: 
21% of students describing a math course said they were very satisfied with their course online 
compared to 15% describing a computer or information science course, 13% for biology courses and 
14% for other STEM subjects, but this difference fell short of statistical significance. Differences in 
typical class size for these different subjects may account for the apparent subject matter effect, but 
subject by class size cell sizes were too small to support statistical testing. 


White students, students from high-income households, students with more frequent internet 
connectivity problems, students in large courses, and those who had not taken an online course before 
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were more likely than their counterparts to say that their STEM course was worse in terms of two 
aspects of the course after it went online (feeling included as a member of the class and keeping them 
interested in the course content). 


Discussion 


Inequalities Unmasked 


A puzzling aspect of the survey data for students from minoritized race/ethnicity groups compared to 
that for students from groups better-represented in STEM is the seeming inconsistency between the 
data on the challenges experienced trying to continue their STEM courses online and that on students’ 
satisfaction with their course experiences and with how well they were learning. Students from 
minoritized race/ethnicity groups were more likely to be adversely affected by technical difficulties, 
conflicts with family and job responsibilities, and feeling too unwell to continue in their course. Hispanic 
students were more likely than others to report having trouble finding help with their course after it 
went online. Yet students from minoritized race/ethnicity groups were not less satisfied with their 
course post-COVID than were students from groups better-represented in STEM. In fact, regression 
analyses controlling for subject area, class size, and gender found that students from minoritized 
race/ethnicity groups were more likely than those from race/ethnicity groups better represented in 
STEM to be very satisfied with the online phase of their course overall (18% versus 13%) and with how 
well they were learning in it (17% versus 13%). 


An examination of race/ethnicity group data for the survey items having students compare different 
aspects of their course experiences before and after the transition to online instruction may offer the 
key to unraveling this apparent paradox. As illustrated in Figure 6 below, students from minoritized 
race/ethnicity groups were less likely than those from non-minoritized groups to judge their post-COVID 
courses as worse than their course experience before COVID in terms of their feeling included as a 
member of the class or opportunities to collaborate with other students. 


Looked at from this vantage point, these data from the COVID experience are revealing weaknesses of 
these aspects of minoritized students’ STEM course experiences before COVID. The remote instruction 
during COVID—widely characterized as ill-planned and a poor substitute for the college academic 
experience—wasn’t worse than business as usual for the majority of students from minoritized 
race/ethnicity groups in many respects. For these students, the experience was as good as, or better 
than, instruction as usual in terms of their feeling that they belonged in the class, their having clarity 
about what they were supposed to do in the course, their sense that the instructor knew their specific 
strengths and weaknesses, and knowing where to get help with the course if they needed it. 
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Figure 6. Aspects of their STEM course experience judged equivalent or better online by student 
race/ethnicity 


Knowing what Minoritized 
was expected 


of you Non-Minoritized 


Instructor's knowledge Minoritized 
of your stregths 


and weaknesses Non-Minoritized 


Availibility of help Minoritized 


on course content 
or About the 

Same Online 
Worse Online 


in| Much Worse Online 


Non-Minoritized i Much Better, Better, 


Feeling included Minoritized 


as a member 


of the class* Non-Minoritized 


Keeping you Minoritized 
interested 


in the course* Non-Minoritized 


Opportunities Minoritized 
to collaborate 


with other students* 


Non-Minoritized 


o 
N 
o 


50 
Percent of Students 
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Implications for Future Instruction 


Despite the overall sharp drop in satisfaction with their STEM courses after they went entirely online 
that students reported, there are some rays of light in the survey data. First, some aspects of courses did 
not get significantly worse after shifting instruction online. Majorities of students were just as confident 
they knew what was expected of them in the course, knew where to get help in the course if they 
needed it, and thought their instructors were as aware of their strengths and weaknesses as they were 
when the course included face-to-face classes. More importantly, the use of a varied set of online 
teaching practices recommended in the literature resulted in higher student satisfaction with the course 
post-COVID. In courses incorporating 6 or more of the recommended online teaching practices, 30% of 
students were very satisfied and 46% somewhat satisfied with their STEM course after COVID. Courses 
incorporating fewer of the recommended practices had lower satisfaction ratings, as described above. 
These are practices that can be promoted and modeled in faculty development activities around 
teaching online. 


Of the eight recommended online teaching practices covered in the survey, the three with the most 
impact on student satisfaction were the inclusion of personal messages to students about how well they 
were doing in the course (courses including this practice had 23% of students very satisfied after COVID 
compared to just 6% very satisfied for courses without it), course activities that asked students to reflect 
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on what they had learned and what they still needed to learn (21% of students in courses with this 
feature were very satisfied after COVID compared to 9% in courses without), and use of real-world 
examples (19% of students in courses with this feature were very satisfied after COVID compared to 7% 
in courses without). Moreover, even the most pervasive post-COVID challenge cited by students— 
maintaining motivation to do well in the course—was mitigated significantly by the practice of sending 
students personalized messages about how they were doing or to make sure they could access course 
materials (59% of students in courses that did not include personal messages found staying motivated to 
be a major challenge compared to only 41% of students in courses that used personal messages). A 
number of courseware products available for STEM subjects incorporate instructor dashboards 
identifying students who are struggling and templates for sending appropriate messages to individual 
students. Activities asking students to reflect on their level of understanding of specific course concepts 
have also become more common in both digital and nondigital instructional materials. 


A recent report from the National Academies of Sciences, Engineering, and Medicine (2018) noted, “a 
growing body of research suggests that the way introductory courses are taught is a significant factor 
that discourages students from continuing in STEM” (p. 117). But research suggests also that involving 
teams of faculty, instructional designers, and faculty development staff in redesigning curriculum and 
instruction can result in a better course and is also a very effective form of professional learning as 
faculty gain insights and know-how from each other as well as from their instructional design and faculty 
development colleagues (Wieman, 2017). 


As colleges and universities work with their faculty to deepen their understanding of how to teach in 
multiple modalities and how to promote active engagement in online classes, a significant portion of the 
training should focus on developing strategies for dealing with inequities across student populations. In 
addition to the continuing need to improve teaching and learning in ways that diminish student 
achievement gaps for low-income students and students of color, institutions will need to take into 
account the challenges associated with learning online that are particularly problematic for these 
students. Instructional designs and course policies and practices need to work for students with poor 
internet connectivity, limited access to appropriate devices, family and job responsibilities, and no quiet 
place at home for doing coursework. 


Everyone hopes that college courses rolling out in fall 2020 and beyond will benefit from having some 
time to shore up their IT infrastructure, design instruction for online delivery, and enhance instructors’ 
online teaching skills. There’s an opportunity to do better by students, and seizing that opportunity is a 
moral imperative and may well be essential for the survival of many higher education institutions. 
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Appendix A: Survey Methodology 


The Digital Promise Survey of Student Perceptions of Remote Teaching and Learning was conducted 
using the nationally representative lpsos KnowledgePanel®, for which participants were randomly 
recruited via address-based sampling to participate in survey research projects online. 


The survey was designed to include approximately 1,000 adults studying at a two- or four-year college or 
university at the undergraduate level, either part- or full-time, who had a class change from including in- 
person class sessions to instruction completely online because of the COVID-19 pandemic. 


Field work was conducted between May 13 and June 1, 2020. Respondents could chose to take the 
survey in either English or Spanish. Starting May 15, email reminders were sent to non-responders every 
two days; those who did not respond by May 29th also received a reminder by telephone, and 
incentives (panel “points” that can redeemed for prizes in future sweepstakes) were increased, first on 
May 27 and then again on May 29. 


Of the 1,709 panelists initially responding, 1,014 were qualified undergraduates taking for-credit college 
courses that started with in-person classes and later went fully online in spring 2020. The median 
completion time of the survey was 10 minutes. Quality control flagged respondents who completed the 
survey in less than one-third of the median time, removing six cases, for a final sample size of 1,008. 


Data were weighted via iterative proportional fitting to the following benchmark distributions of adult 
undergraduate students with less than a bachelor’s degree from the 2018 American Community Survey: 


e Gender (male, female) by age (18-20, 21-22, 23-29, 30+) 
e = Race/ethnicity (White, Black, Other, Hispanic, 2+ races) 
e Census region (Northeast, Midwest, South, West) 
e Education (high school, some college, associate’s degree) 
e Household income (less than $25,000, $25,000-$49,999, $50,000-S$74,999, $75,000-S99,999, 
$100,000-$149,999, $150,000+) 
e Language proficiency (English-proficient Hispanic, bilingual or Spanish-proficient Hispanic, non- 
Hispanic) 
e Hispanic nativity (U.S.-born, not-U.S.-born, non-Hispanic) 
e School type (public, private) 
Weights were trimmed at 0.23 percent and 99.77 percent and then rescaled to match the sample size of 
all respondents. The final weights for qualified respondents (i.e., those who had a class change from in- 
person to completely online) were extracted from the above weights and re-scaled to sum to the sample 
size of qualified respondents. The average design effect for the final weight is 1.35, for a margin of error 
of 3.6 percentage points for the full sample. 


Survey respondents were asked if they were taking a course in a STEM field and if so to answer 
subsequent survey items with respect to their STEM course (or to the STEM course they considered 
most consequential for their future if they were taking more than one). Students were asked to indicate 
the subject area of the course they were describing using a pull-down menu of options with an open- 
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response alternative. In some cases, students indicated they were taking a STEM course but wrote ina 
course subject that was not STEM (for example, history). In other cases, students said they were not 
taking a STEM course but wrote in a course falling within a STEM field (for example, college algebra). 
Analysts checked student course subject descriptions and back-coded course descriptions as STEM or 
non-STEM using the list of STEM subjects shown in Table A-1. After back-coding, the application of 
sampling weights produced a sample of 638 qualified respondents describing STEM courses. 


The margin of error was 4.6 points for students reporting on STEM courses. Error margins are larger for 
subgroups. 


Table A-1. Subject areas coded as STEM 


Subject Area 


Statistics or Data Science 
Other: [TEXT BOX] 


Table A-2 shows characteristics of the sample of students taking STEM courses compared to the full 


survey sample of 1,008 and to population data on college undergraduates from the American 
Community Survey (ACS). 
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Table A-2. Characteristics of subset of national sample for survey taking STEM courses before and 
after applying sample weights 


STEM 
STEM Sample Full Sample Sample Full Sample 
Variable Unweighted Unweighted Weighted Weighted 


% Hispanic 


% non-Hispanic 
White 

% Asian, Other, or 
Multiple 


% lower income* 


*Household annual income below am 000. 


Missingness in data was examined. For items viewed by all participants, missing data (i.e., skipped or 
refused items) accounted for just 0.7% of all data, resulting in 99.3% data coverage. For any one item, 
missingness ranged from 0% to 1.6%. The two items with missingness above 1.5% were examined with 
univariate tests to determine if any patterns of missingness emerged. There were no significant 
differences on response for either item related to variables of race/ethnicity, income, gender, age, or 
location following transition to remote instruction (rural, urban, or suburban). Given the low rate of 
missingness and the lack of significant differences between student groups who skipped or did not skip 
items, pairwise deletion (removing a respondent’s missing item response from the analysis for that item 
but not removing the respondent from the analysis of other items) was used to maximize the data 


available for analysis. 
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Appendix B: Survey Items and Response Distribution 


Survey of Student Perceptions of Remote Teaching and Learning — STEM 
Student Data 


This survey was conducted May 13—June 1, 2020, among a random national sample of 1,008 full- or 
part-time students enrolled in a two- or four-year college or university who took in-person classes or 
lectures before the coronavirus outbreak began that were then transitioned completely online. The 
results below are among the sample of 620 students who took a course in science, technology, 
engineering, medical/health science or mathematics that moved online. Results have a margin of 
sampling error of 4.6 points among these students. 


The survey was produced for Digital Promise by Langer Research Associates of New York, N.Y., with 
sampling and data collection by Ipsos Public Affairs via its online, probability-based KnowledgePanel®, 
which provides internet access to randomly recruited participants. The response frequencies below 
were produced by Langer Research Associates. Due to the nature of missingness described in Appendix 
A, percentages and statistical analyses for survey items above used only complete data in weighted 
samples, so distributions of responses outlined below will differ very slightly from the percentages in 
this report. 


Screener Questions 


Screener 1. Have you taken one or more college courses for credit this year (since 
Jan. 1, 2020)? 


Screener 2. [IF YES] Did any of these courses include in-person classes or lectures 
before the coronavirus outbreak began, and then go completely online? 


Screener 3. Were any of your courses that moved completely online in science, 
technology, engineering, medical/health science or mathematics, known as STEM 
subjects? 


[IF ONE STEM COURSE] For the rest of these questions, please think about the STEM 
course you took that moved online this year. 


{IF MORE THAN ONE STEM COURSE] For the rest of these questions, please think about the 
STEM course you took that moved online this year that was most important to you in 
terms of your future goals. (If multiple STEM courses were equally important, pick one 
at random.) 


1. What was the subject of this course? 


STEM course NET 100 
Biology 17 
athematics 15 
Computer Science or Technology 13 
Engineering or Engineering Technician 12 
Chemistry 10 
Health Professions including Nursing 10 
Statistics or Data Science 5 
Environmental Science 4 
Physics 4 
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Information Science or Technology S 
Geology 1 
Agricultural Sciences s 
Astronomy s 
Other: STEM course 4 
Skipped 0 


ae 


Thinking back to when this course included in-person classes, before it moved 


completely online, how satisfied or dissatisfied were you with the course overall? 


Si 


=== Satisfied ----- --- Dissatisfied ---- 
NET Very Somewhat NET Somewhat Very Skip 
87 47 40 13 11 2 0 


After this course moved completely online, how satisfied or dissatisfied were you 


with the course overall? 


==Si=5 Satisfied ----- --- Dissatisfied ---- 
NET Very Somewhat NET Somewhat Very Skip 
a7 15 42 42 27 1S aE 


Were you satisfied or dissatisfied with each of these aspects of the course after 
moved online? 


The quality of the instruction 


==Sea Satisfied ----- --- Dissatisfied ---- 
NET Very Somewhat NET Somewhat Very Skip 
66 25 41 34 24 10 x 


The quality of the course content 


----- Satisfied ----- --- Dissatisfied ---- 
NET Very Somewhat NET Somewhat Very Skip 
70 26 44 30 23 7 x 


The instructor’s preparation 


aS Satisfied ----- --- Dissatisfied ---- 
NET Very Somewhat NET Somewhat Very Skip 
74 35 39 26 19 7 % 


How well you felt you were learning overall 


rasan ane Satisfied ----- --- Dissatisfied ---- 
NET Very Somewhat NET Somewhat Very Skip 
54 14 39 46 28 18 x 


Compared with the course when it included in-person classes, was the course after 
moved completely online better or worse on each of these items? 


Your feeling included as a member of the class 


ar Better -====->-== Spo Sc >>, Worse: 2s >45-5= 
Much better Better About Worse Much worse 
NET online online the same NET online online Skip 
it 3 4 39 53 37 16 1 
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Opportunities to collaborate with other students on course work 


SSsSSS-=5 Better =-=<<-==== ss5-=<=-= Worse =====-=== 
Much better Better About Worse Much worse 
NET online online the same NET online online Skip 
8 3 5 25 66 38 28 1 


The instructor's knowledge of your strengths and weaknesses 


aSSS=5S= > Better =<s====== SSSss5554> Worse. =S=====5= 
Much better Better About Worse Much worse 
NET online online the same NET online online Skip 
7 3 4 49 43 29 14 1 


The availability of help with the course content 


Sooo se Better “->--=-5-== Soss>-5>, Worse: === >-s>-— 

Much better Better About Worse Much worse 
NET online online the same NET online online Skip 
14 4 10 41 44 33 11 1 


Understanding what was expected of you in the course 


SS SSSSS== Better --------- ssSs=s=>= Worse ==<s-==-=-= 

Much better Better About Worse Much worse 
NET online online the same NET online online Skip 
12 4 9 51 36 26 10 at 


Keeping you interested in the course content 


SSSSSs555 Better --------- SsSsaS+=== Worse -==4=+=<> 
Much better Better About Worse Much worse 
NET online online the same NET online online Skip 
9 3 6 31 59 37 21 1 


After this course moved online, did it include any of the following? 
Live sessions in which students could ask questions and participate in discussions 


Included Did not include Skip 
70 29 1 


Live lectures by the instructor, with the class watching in real time 


Included Did not include Skip 
60 39 1 


Recorded lectures 


Included Did not include Skip 
70 29 1 


Assignments to meet in “breakout groups” during a live class 


Included Did not include Skip 
23 75 2 
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e. Assignments to work on group projects separately from the course meetings 


Included Did not include Skip 
33 65 2 


f. Breaking up class activities into shorter pieces than in the previous in-person 
course 


Included Did not include Skip 
32 67 1 


g. The instructor using examples from the real world to illustrate the course content 


Included Did not include Skip 
64 34 1 


h. Frequent quizzes or other assessments 


Included Did not include Skip 
68 31 1 


i. Assignments having you express what you had learned and what you still needed to 
learn 


Included Did not include Skip 
46 52 1 


j. Personal messages from the instructor about how you were doing in the course or to 
make sure you could access course materials 


Included Did not include Skip 
50 49 1 


k. Pre-recorded videos from external sources (e.g., YouTube) 
Included Did not include Skip 
5D 44 1 


7. In your own words, what would you say was your greatest challenge in learning from 
this course after it went online? 


8. Were these a problem for you, or not a problem, in taking this course after it 
moved online? 


a. Fitting the course in with your home/family responsibilities 


==35> Problem ----- 
NET Major Minor Not a problem Skip 
56 18 38 44 * 


b. Fitting the course in with your work schedule, if you work for pay 


=S55e Problem ----- 
NET Major Minor Not a problem Skip 
31 8 23 69 ay 
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c. Feeling too unwell, physically or emotionally, to participate 


SSSs= Problem ----- 
NET Major Minor Not a problem Skip 
45 14 32 54 1 


=SS== Problem ----- 
NET Major Minor Not a problem Skip 
58 18 40 41 1 


e. Finding a quiet place where you could do the course online 


ale Problem ----- 
NET Major Minor Not a problem Skip 
56 21 35 43 it 


SSsS= Problem ----- 
NET Major Minor Not a problem Skip 
82 45 37 17 * 


9. What kind of computing device did you use most often for accessing this course 
after it moved online? 


SS=S55-—+2> Computer ---------- 
NET Desktop computer Laptop Tablet Smartphone Skip 
95 16 80 3 2 7 


10. [IF ANSWERED Q10] Which best describes this device: 


SSSS5—S>—---7- Personal device --------------- 


A personal device A personal device you A shared device 
NET only used by you shared with others in a public place Skip 
99 92 7 7 1 


11. [IF PERSONAL DEVICE SHARED WITH OTHERS] How many people did you share this device 
with? 


Insufficient sample size (n = 48). 


12. [IF ANY PERSONAL DEVICE] Was this device: 


Owned by someone else 
Owned by you Provided by the school (other than the school) Skip 
92 4 4 1 


13. How did you usually get internet access for this course after it moved online? 


Internet access 


Internet access at or near your Public access 
you had at home home provided Internet access in another 
already by the school at the school location Skip 
95 1 2 1 1 
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14. In accessing this course after it moved online, how often, if at all, did you 
experience serious internet connectivity problems that interfered with your ability to 
attend or participate? 


Ss lteriteietenien Often --------- --- Rarely/never --- 
NET Very often Often Occasionally NET Rarely Never Skip 
LS 6 9 31 54 37 18 0 


15. How often, if at all, did you experience serious hardware or software problems 
that interfered with your ability to attend or participate in this course after it 
moved online? 


Seta taae Often --------- --- Rarely/never --- 
NET Very often Often Occasionally NET Rarely Never Skip 
7 2 5 16 75 35 40 2 


16. During the period that this course was being held online, did any of these get 
better or worse over time? 


a. The quality of the course overall 


Got better over time No change Got worse over time Skip 
18 55 27 x 


b. Technical problems 


Got better over time No change Got worse over time Skip 
21 67 11 x 


c. The instructor’s online teaching skill 


Got better over time No change Got worse over time Skip 
29 55 16 bg 


d. Your satisfaction with the course 
Got better over time No change Got worse over time Skip 
16 53 31 0 
17. In your opinion, which of these do you think was more responsible for any problems 


you may have experienced with the course after it moved online? 


The problems mostly The problems mostly I did not experience 


reflected the reflected inherent any problems with 
unplanned move of limitations of the course after it 
the course online online learning moved online Skip 
47 39 13 1 


18. When this course was held in person, how were you taking it? 


For a letter grade Pass/fail Skip 
92 8 * 


39 


19. [IF FOR A LETTER GRADE] When this course moved online, was it... 


Your choice of 
Graded, with no option Pass/fail, with no whether to take it 


for pass/fail option for a grade graded or pass/fail Skip 
33 7 60 % 
18/19 NET: 
<== Initially taking class for letter grade ------ Initially 
Graded, with Pass/fail, taking 
no option with no option Your Skip. class Skip 
NET for pass/fail for a grade choice Q20 pass/fail Q19 
92 30 6 56 * 8 * 


20. [IF PASS/FAIL WITH NO OPTION FOR LETTER GRADE IN Q20] How, if at all, did this 
change in the grading system affect your motivation to work hard in the course? 


Insufficient sample size (n = 31). 


21. About how many students were enrolled in this course? 
SSSSSSSS= More than 75 -------- 


Fewer than 35 35 to 75 NET 76 to 125 More than 125 Not sure 
50 28 21 10 11 0 


22. What kind of school offered this course? 
4-year college 2-year or community 


or university college Other Skip 
70 29 1 0 


23. Is this a public institution (like a county or state school) or a private 
school/institute? 
Public Private Skip 
84 16 0 


24. Which best describes your course of study:* 


6/1/20 
I am majoring in a STEM subject 63 
I am majoring in a different subject, not STEM 29 
I have not declared a major 5 
Does not apply 4 
Skip 0 


*Respondents could select both STEM and non-STEM major 


25. Before this March, had you previously taken any college courses completely online? 
SSSs2sS2—5 SSS asS5-S54-=5> VS) Sss5SS5SSS255 SS ssS25-s5> 


No, none NET Yes, one Yes, two to three Yes, four or more Skip 
43 57 20 24 13 1 
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26. Will you graduate before the fall semester starts? 


-- Graduated or will in the summer - 
Yes, I will No, I will not 


Yes, I graduated graduate in graduate before Does not 
NET this semester the summer the fall semester apply Skip 
U7 11 6 719 3 1 


27. [IF NOT GRADUATING BEFORE FALL SEMESTER OR DOES NOT APPLY] Looking ahead to next 
fall, how likely are you to take another course from the same institution if all its 
courses are offered online only, at the same cost as in-person courses? 


=—S55— Likely ------- ----- Unlikely ------ 
NET Very Somewhat NET Somewhat Very Skip 
70 44 26 29 15 14 A 


28. [IF NOT GRADUATING BEFORE FALL SEMESTER OR DOES NOT APPLY] Compared with an 
in-person course, what do you think is a fair price to pay for a completely online 
course from this institution? 


6/1/20 
Would take another completely online course at 
this institution NET 94 
The same price (100%) as the in-person course ¥ 
Three-quarters (75%) of the price of the in-person course 29 
Half (50%) the price of the in-person course 49 
One-quarter (25%) of the price of the in-person course 10 
Wouldn't take another completely online course at this 
institution regardless of price 5 
Skip 1 


29. If this institution offers online courses next fall, how do you think they will 
differ, if at all, from your experience this spring? Do you expect online courses in 
the fall to be... 


SSsssss Better ------ About the s===== Worse ‘======== 
NET Much Somewhat same NET Somewhat Much Skip 
49 10 40 40 11 7 3 * 


30. While you were taking this college course completely online, did you also work for 
pay? 


Yes No Skip 
46 54 * 


31. [IF WORKED FOR PAY] How many hours per week did you usually work for pay while you 
were also taking this college course completely online? 


SSS S5 55 -Ss= > Number of hours ------------- 
1-10 11-20 21-30 31 or more Skip Mean Median 
26 30 22 22 iL 21.8 20.0 


32. While you were taking this college course completely online, were there any 
children under age 18 living with you? 


Yes No Skip 
33 66 1 
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33. [IF LIVING WITH ANY CH 
that child/those children? 


Yes No Skip 


33 67 * 
32/33 NET: 
No children 
under 18 NET 
66 33 


**k*k END *** 


ILDREN UNDER 18] Were you responsible for providing care for 


=e Living with children under 18 ------- 
Responsible Not responsible Skip Skip 
for care for care Q34 Q33 

11 22 * 1 
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